Kog AI

Kog AI achieved over 3,000 output tokens per second per request for an FP16 2B model on a single 8× MI300X node using a monokernel that eliminates per-token kernel launches. The technique collapses th…

Kog AI launched a tech preview of its Kog Inference Engine (KIE), achieving 3,000 output tokens per second per request on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 GPUs using a 2B model. The comp…

The Kog AI team implemented a single-kernel LLM inference engine on AMD MI300X GPUs, achieving over 3,000 output tokens per second per request for a 2B-parameter model in FP16 precision. The monokerne…